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Abstract 

Quantum mechanics is derived from the principle that the universe contain as 
much variety as possible, in the sense of maximizing the distinctiveness of each sub¬ 
system. 

The quantum state of a microscopic system is defined to correspond to an ensem¬ 
ble of subsystems of the universe with identical constituents and similar preparations 
and environments. A new kind of interaction is posited amongst such similar sub¬ 
systems which acts to increase their distinctiveness, by extremizing the variety. In the 
limit of large numbers of similar subsystems this interaction is shown to give rise to 
Bohm's quantum potential. As a result the probability distribution for the ensemble 
is governed by the Schroedinger equation. 

The measurement problem is naturally and simply solved. Microscopic systems 
appear statistical because they are members of large ensembles of similar systems 
which interact non-locally. Macroscopic systems are unique, and are not members 
of any ensembles of similar systems. Consequently their collective coordinates may 
evolve deterministically. 

This proposal could be tested by constructing quantum devices from entangled 
states of a modest number of quits which, by its combinatorial complexity, can be 
expected to have no natural copies. 
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1 Introduction 

This paper presents a new completion of quantum mechanics based on three key ideas. 
The first is that quantum mechanics is necessarily a description of subsystems of the uni¬ 
verse. It is an approximation to some other, very different theory, which might be applied 
to the universe as a whole [H[2i[3j]. 

The second idea is that the quantum state refers to an ensemble of similar systems 
present in the universe at a given time. We call this the real ensemble hypothesis ing . By 
similar systems we mean systems with the same constituents, whose dynamics are subject 
to (within errors that can be ignored) the same Hamiltonian, and which have very similar 
histories and hence, in operational terms, the same preparation. The very peculiar idea 
underlying this proposal is that such similar systems have a new kind of interaction with 
each other, just by virtue of their similarities. This interaction takes place amongst similar 
systems, regardless of how far apart they may be situated in space, and thus, if these ideas 
turn out to be right, is how non-locality enters quantum phenomena. 
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This hypothesis is motivated by a line of thought involving the application to quantum 
gravity and quantum mechanics of some very general principles. But in the interest of 
getting quickly to the point, this motivation is postponed untill section 6. We will only 
say here that quantum gravity points to the possibility that space and locality are both 
emergent and that, if this is so, we should expect there to be defects in locality, where 
events are connected which are far separated in the emergent low energy classical metric. 

If the reader will take this real ensemble hypothesis as a provisional idea, he or she will 
see that, together with the third idea, it leads to quantum mechanics. 

The real ensemble hypothesis was explored earlier in H21i , where a new kind of inter¬ 
action amongst the similar systems which make up the ensemble was posited and shown 
to yield quantum dynamics. However that work could be criticized because the particu¬ 
lar inter-ensemble interaction was complicated and motivated only by the fact that it gave 
the right answer. 

A similar proposal was made in Ii22l [231 , where the idea of many interacting classical 
worlds (MIW) was introduced to explain quantum mechanics. This also posits that the 
quantum state refers to an ensemble of real, existing, systems which interact with each 
other, only those were posited to be near copies of our universe that all simultaneously 
exislQ. 

The third key idea is that the inter-ensemble interaction can be related to the an ob¬ 
servable called the variety of the collection of similar subsystems. This is also motivated 
by the general principles, as will be explained in section 6. 

The principle of maximal variety, was formulated with Julian Barbour in the 1980's HTOl . 
The variety of a system of relations, V, is a measure of how easy it is to distinguish the 
neighbourhood of every element from that of every other. 

The basic idea can be applied to every system with elements e ir labeled by i — 1,..., N 
whose dynamics depends on relational observables, X t j . The elements could be particles 
or events or subsystems and the relations could be relative position, relative distance, 
causal relations, etc. We proceed by defining the view of the z'th element, this summa¬ 
rizes what element i may "know" about the rest of the system by means of the relational 
observables. The view of i is then denoted Vf X^). 

Different systems will be described by different views. For example, if the system is N 
points in a d dimensional Euclidean space, the view of the A 'th point is the list of vectors 

1 If the ontology posited by the Il22ll23| papers may seem extravagant, their proposal had the virtue of a 
simple form for the inter-ensemble interactions. This inspired me to seek to use such a simple dynamics in 
the real ensemble idea. In particular, an important insight contained in Ii23l is that if there are N particles 
on a line with positions, Xi, with i 1 ,N, the density at the fc'th point can be approximated by 

^ (i) 
This motivates the choose of a ultraviolet cutoff, in equation l25t below. 
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to the other points, weighed by the distance. 


t fka _ 

* ~ D(i, k) 2 


tX 


X i X k I 2 


( 2 ) 


We then define the distinctiveness of two elements, i and j to be a measure of the differ¬ 
ences between the views of i and j. If the views live in a vector space this can be denoted, 

Tii = \ Vi-Vjl 2 ( 3 ) 


The variety is then defined to measure the distinguishability of all the elements from each 
other. 

V = N(N-l) ^ ^ (4) 

The distinctiveness provides a metric on the set of subsystems. 

hij = lij = \Vi — Vj\ 2 ( 5 ) 


This metric tells us that systems are close if they have similar views of their relations to 
the rest of the universe. For example, two events that are close in space will have similar 
views. 

But this is not the only circumstance in which two events may have similar views. If 
the events are each part of the history of a microscopic system, that can each be considered 
to be effectively isolated, and if those two isolated systems have similar constituents, 
environments and preparations, than their views may also be similar. 

Now assuming conventional notion of locality, the degrees of freedom at nearby events 
can interact. But suppose locality in space is not primary. Suppose, instead, that the met¬ 
ric on the space of views is actually what determines the relevant notion of locality for 
interactions. As a consequence, two systems may interact when they are nearby in space, 
or when their views are similar because they have the similar constituents, environments and 
preparations. The former give conventional local interactions, while the latter case gives a 
new kind of interactions. The aim of this paper is to show that the latter kinds of interac¬ 
tions may be responsible for quantum phenomena. 

In particular, we will show that when these new interactions amongst members of 
the ensemble of systems with similar views acts to increase the variety of the beables in 
that ensemble, they give rise to the quantum potential of Bohmian quantum mechanics. 
This can be made plausible if we consider the fact that the Bohmian potential is repulsive 
and so acts to smooth out the wave function, giving rise to a greater variety of beables 
represented in the ensemble. 

There are then three basic hypotheses in this paper. 


1. In the microscopic causal geometry underlying nature, two systems can interact if 
they are within a distance R in the metric h %3 . There are two ways this can happen. 
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It can happen when they are nearby in the emergent macroscopic notion of spatial 
geometry When two people stand next to each other and scan a landscape they 
see similar views. But two microscopic systems can also be very far apart in the 
macroscopic geometry and still have a similar view of their surroundings. When 
this happens there are a new kind of interactions between them. 

2. Similar systems, nearby in hij but distant in space, form ensembles that mutually 
interact. It is these ensembles that the quantum state refers to. 

3. These new interactions are defined in terms of a potential energy which is propor¬ 
tional to the negative of the variety. Lower energy implies higher variety. 

We will see that these new interactions between members of the ensembles that define 
quantum states give rise to quantum phenomena. 

The main result of this paper is that when N, the number of subsystems in the ensem¬ 
ble, is large, the variety can be expressed in terms of a probability density for the ensemble 
and that, when so expressed, V is closely related to the quantum potential of Bohm. Con¬ 
sequently the evolution of the ensemble probabilities is given by the Schroedinger equa¬ 
tion. A second result is a prediction of specific corrections that arise and are expressed as 
non-linear corrections to the Schroedinger equation. 

Another result is that the measurement problem is naturally and simply solved. Mi¬ 
croscopic systems appear statistical, when described as local systems in isolation, because 
they are members of large ensembles of similar systems which with they interact non- 
locally. Macroscopic systems are unique, and are not members of any ensembles of sim¬ 
ilar systems. Not being members of large ensembles, their collective coordinates are not 
disturbed by non-local interactions with distant subsystems, nor can they be described by 
quantum states. Consequently their collective coordinates may evolve deterministically. 

The precise proposal for a non-local completion of quantum mechanics is presented 
in section 2, while section 3 is devoted to deriving quantum mechanics as an approxima¬ 
tion in the limit that N, the number of subsystems in the ensemble goes to infinity. In 
section 4 we discuss several experimental tests that become possible in cases where N is 
small, while section 5 considers several possible objections to these results. Then finally, 
in section 6, we discuss in more detail the motivation for this proposal. 


2 The dynamics of extremal variety 

We now begin the formal development, by which we derive quantum mechanics from 
the principles we have described. 

2 Some possibly related approaches are 1321 . 
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We consider an ensemble of N identical systems, each of which lives in a configuration 
space we will for simplicity take to be R d , coordinatized by x\, where a = 1,... d and 
k = l,...,N. 

All the relational information about a subsystem of the universe is contained in the 
view that subsystem has of the rest of the universe, through its causal links or other rela¬ 
tions to other subsystems. This is the central element we will employ in our reconstruc¬ 
tion of quantum theory 

Let us then define the view of the i'th system, 


_ rpCt 

t rka _ ,h i _k 

* ~ D(i,k) 2 


Xi 


XI 


I x i x k 1 2 


(~)(R — |x“ — x 


( 6 ) 


for k f i, be seen, for each i, as a vector of components labeled by k, each component of 
which is a vector, that shows the system i's relations to its N — 1 neighbours. The vector's 
components are weighed by the distance. This can be called the view of the rest of the 
system, experienced by the i'th element. The closer k is to i, the larger is Vf and the more 
important k is to i'th view of the rest of the system. 

Note that we insert a cutoff R on the view, by choosing the distance function to be 


D(i, k) 2 


\x ; 


x 


a 1 2 


0(R- \x a i - x a k \) 


(7) 


Thus if the i 'th system is more than a metric distance R from the i'th system it "falls 
outside the horizon" and is an infinite distance away. The cutoff R will play a role in 
what follows. 

Now let us construct a measure of the differences between two elements i and j. We 
can simply take the difference of the two vectors, Vf and Vf. 

T >>= <8) 
k 


can be called the distinctiveness of i and j. The larger X l3 are the more easily they can be 
differentiated by their views. 

To get the variety we sum this over all the pairs i f j 

i^j i^j k 

where A is a dimensionless normalization constant. 

So we define a new inter-ensemble potential energy as 

u v = -fv ( 10 ) 
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We choose to posit that the potential energy is the negative of the variety so that in the 
ground state the variety will be maximized. Note that V has dimensions of inverse- 
length-squared and that this potential is negative definite. The constant — is necessary 
for dimensional considerations, to turn an inverse area into an energy. Of course an h 
is required if we are to make good on our claim that these new interactions give rise to 
quantum phenomena. 

We can think of V in a different way, as a local function in X}., by reversing the order 
of the summations. 

V = |E V * = ^EEW-Cf (u> 

2 k ij=j 

where 

= E (V? a -V*‘f (12) 

i+k+j 


2.1 The fundamental dynamics 

To write a dynamical theory we need to introduce momenta beables p\, in addition to the 
position beables 

However the correspondence with quantum mechanics requires that in the large N, 
continuum limit, the momenta of the particles, p' a , merge into a momentum density p a (x). 
This, moreover, must be a gradient of a phase S(z), which comes from the decomposition 
of the wave function. 

p a (x a ) = d a S(x a ) (13) 

where S is related to a complex phase, w{x a ) = e% s subject to 

w*(x a )w(x a ) = 1 (14) 

Consequently, Takabayashi and Wallstrom |f27l l26ll noted that further conditions are 
needed to guarantee that e lS ^ h is single valued. We can address this by replacing the 
momenta beables, v\ with a complex phase factor beable (one for earh subsystem), asso¬ 
ciated, 

Wi, w*Wi = 1 (15) 

We can write 

Wk = e^ Sk (16) 

but remember that is only defined modulo 2nh. 

Mindful that we want to express the theory symmetrically in all pairwise relation¬ 
ships, we will posit that the momenta p^ are composite variables which code a subsys¬ 
tem's view of the ratios of the phase factor beables. 

^=-4e^(5) 
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So we write the kinetic energy as 


_ zn 2 ^ 1 

K E — IZe - 7 - xxx 

• • 2m7V 2 (x k - xj) 2 [ \w k J J 


In I — 


(18) 


where Z is a normalization constant to be determined. 

Putting this together with the inter-ensemble potential energy we have the fundamen¬ 
tal action. 


S(w , x ) = 


J k l jyfc 


In | — 

Wk 


H[x, w] 


(19) 


where 




Tie In [ ^ 

w k 


A 

In 




k i+j 


( 20 ) 

Here Z. Z 0 , and /I are normalization constants and U(x) is an ordinary potential energy 
Note that we have used the relative locality form for the symplectic potential fl7l . 


s„ = -z 0 J = - z » / 


k j^k 


k j^k 


In | ^ 

w k 


( 21 ) 


Our task is now to show that when N is large this is equivalent to ordinary quantum 
mechanics. 


3 Derivation of quantum mechanics 

We first evaluate the inter-ensemble potential energy, then we do the same for the average 
of the kinetic energy and the symplectic potential. 


3.1 The origin of the quantum potential 

To express < V > as an integral over local functions we write, for a function o(x), 


1 

N 


< <t> >= — Y. 4>{x k ) -» / d d zp{z)4>(z) 


( 22 ) 


In the limit N —* oo this defines the probability density for configurations, p(z). 
Similarly we turn the sums on i to an integral. 


1 . f R 

-^y'j>(xk + i,x k ) -» Z J d d xp{z + x)(f){z + x, 


(23) 
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and similarly for j. The possibility that the integral only approximates the sum for finite 
N, because of the roughness of the estimate for the limits on the integral, is accounted for 
by an adjustable normalization factor Z. 

Note that we have to be careful to impose limits on the integral to avoid unphysical 
divergences in A These divergences are unphysical because for finite N two configuration 
variables, x% and x“, cannot come closer than a limit which varies inversely with the 
density at x n k and N. This is because if x k and x" are nearest neighbours in the distribution, 
the density at one of their locations is related to their separation. 


pK) ~ 


1 

N\x k - Xj 


(24) 


Hence, for finite N they are very unlikely to coincide. When we approximate the sums by 
integrals, the integrals representing intervals between configurations must then be cut off 
by a short distance cutoff a that scales inversely like a power of Np{z). The short distance 
cutoff a(z) on the integral above in d d x then expresses this fact that there is a limit to 4 
related to the density. Hence the short distance cutoff is at 


{Np(z))d 


(25) 


There is also an infrared cutoff, R coming from 0. This tells us that two systems further 
than R in configuration space do not figure in each other's views. A key question turns 
out to be how the physical cutoff scales with N. We will define 


r' = N^R 


(26) 


to represent a fixed physical lengths scale which is held fixed when we take the limit 
of large N at the end of the calculation. That way, the physical ultraviolet and infrared 
cutoffs scale the same way with N. But the large scale, infrared cutoff, r' can't know about 
the value of the probability distribution at some far off point z, so while a scales with p, r' 
doesn't. 

As a result when we scale x and d d x with a to make the integrals dimensionless, we 
define r, such that, R = ar. But we then hold fixed r' = -Ar = N^R as we take N large. 

pH 

r', unlike r, is a length. We shall see that r' defines a new physical length scale at which 
the linearity of quantum mechanics gives way to a non-linear theory. 

Thus, the continuum approximation to the variety is, 

V = [ d d zp{z)Z v [ d d x f d d y [(^ ^ f p(z + x)p(z + y ) (27) 

J Ja J a x " V 

We do a scale transformation by writing x a = aa a and y a = aj3 a . To get a single integral 
over a local function we can expand 

p(z + aa ) = p(z) + aa a d a p(z) + ^ a 2 a a a b d‘l b p{z) + ... (28) 
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and similarly for p(z + a/3) and perform the integrations, holding the upper limit r' fixed. 
The normalization factor is 


d 2 N 2 


Z v = 


d 2 


The result is 


V = 


N 3 fl 2 (r d -1) 2 2Nfl 2 r 2d 
d 


d ‘ zp ( T _ (UpY + d d + 2 


r »w + 


(29) 


(30) 


p Nd a + 2 p* 

Here we ignore total derivatives, which don't contribute to the potential energy. The first 
term is an ignorable constant. The second term is what we want; its variation gives the 
Bohmian quantum potential. 

The higher order terms are suppressed by powers of The result is 


Nd 


u v = -//-v = //- [ d d zp(-d aP ) 2 + o(-T) 

8 m 8m J p N a 


(31) 


which we recognize as the term whose variation gives the Bohmian quantum potential. 
The leading correction is 


pj 2 

u av = __2_AV = 

8 m 


1 h 2 r' 2 f , .1 2 . 2 

' d d zp(-V~p) 2 


Nd 8m j p 

which contributes non-linear corrections to the Schroedinger equation. 


(32) 


3.2 The kinetic energy 

We can similarly evaluate the kinetic energy. We write the continuum approximation, 
using a function w(x) defined so that 


iu(x k ) = w k 


(33) 


This is possible because the inter-ensemble interaction is repulsive so it would require 
infinite potential energy for two configurations to sit on top of each other. So there are 
never two members of the ensemble k and j such that x'/ = x". Thus, if there is a member 
of the ensemble sitting at a point x" then it is unique and we can assign a definite w(x k ) = 
w k to it. 

We find using (1181221) 


K E - = ne ^ f d d zp{z)Z KE d d xp(x) (z ^ x y 2 


In 


w(x] 


pR 


= 2^/^W 


d d xp(x)Z KE 


(z — x) d 


w(z) 

[S(x) ~ S(-)f 


(34) 

(35) 
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= Ph,S( x ) 


where we recall that, w(x) = e 

We rewrite in terms of act = x — z and expand in powers of a = —— r . We find 


(Np) <Z 

j-2 p pr 

K.E. = — I d d z^(z)Z KE I d d aa d - 2 (p(z)+aa a d a p+...)(^a a d a S + ...) 2 (36) 

i ir d — ])^z KE 


2m 


dr z 


p(o a sy 


A 


+ 

Nd 


We now set the normalization constant to extract the kinetic energy 


Zre = 


A 


(37) 


(38) 


( r d — l)f2 

After which we take the limit r' large, followed by A —» oo. We find the renormalized 
kinetic energy is 

\d a sy 


K.E. = / dzp(z) 


2m r 

Putting this together with the above results we find 


+ °d> + +°(^) 


H = / d d zp(z) 


+ V + O(-) + 0(h 
2m am p r A 


(39) 


(40) 


3.3 The symplectic measure 

The last step is to derive the continuum approximation to 

S°(w,x ) = Z 0 f dt^p k a x a k = -Z 0 f dt^2 

k k 

where, inspired by relative locality[fT7|. we integrate by parts in dt. The velocity of the 
momenta p k are expressed in terms of the beables as 

1 


Pa*! 


(41) 


Pa = ~ h E [-A“ (Sj - S t 




The continuum approximation to this is 

S° —» —A f dt f d d zZ 0 p(z)z a p a (z) 


where 


Pa{Z) = 


d d x —7 ( p(z + x)S(z + x) — p(z)S(z) 
x 2. 


d d x^rtx c d c ( p{z)S{z ) + ... 


f \r d — 1 ) 
dNp(z) 


d a ( p(z)S(z ) 


(42) 

(43) 

(44) 

(45) 

(46) 
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Consequently, with 


(47) 


Z n = 


dN 


we find 


fl(r d — 1) 

S° = J dt J d d zp(z)S(z ) 

Putting the three pieces together we have 

S — f dt [ d d zp(z ) 


s + + Z ( - daP y + u + 0(1) + 0(h 

2m 8 m p r JM 


(48) 


(49) 


3.4 Recovery of quantum mechanics 

The action (149)) has two equations of motion which arise from varying with p and S. 
These are the probability conservation law 


p(z“) = d a (p-g ab d b S(x a )) 


m 


(50) 


and the Hamilton Jacobi equation, with the addition of the quantum potential term 

1 


- S =^9 a \^)(^) + u + uQ 

2 m ox n axh 


(51) 


where the quantum potential U Q is given by 


[rQ ft 2 

(52) 

2m yfp 

These are nothing but the real and imaginary parts of the Schroedinger 

equation, for 

V(x,t) = y/pw = y/pe* s 

(53) 

which we have thus shown satisfies 


r/T / h 2 9 \ 

(54) 

ih— = -V 2 + U)V 

dt \ 2m J 


To complete the derivation of quantum mechanics, let us draw an important distinc¬ 
tion between p(z), the probability distribution for the ensemble of N systems and pk(xk), 
which is the probability distribution for a single subsystem in the ensemble. We conjec¬ 
ture that over time the non-local inter-ensemble interactions (ITOl) randomize the trajecto¬ 
ries of the individual elements so that over some convergence time, r, for each k, 

p k (x) -> p{x). (55) 

Standard arguments would suggest that this is true, but it has not been shown. 
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4 Experimental tests 

The framework proposed here is highly vulnerable to experimental test. There are several 
kinds of tests possible in which departures from predictions of conventional quantum 
mechanics may be searched for. 


• N — 1. Systems which have no causally indistinguishable copies in the universe 
are expected to behave classically, because for such systems there is no confusion 
possible and no quantum potential. Macroscopic systems with many incoherent 
degrees of freedom will be of this kind and, as already remarked on, this solves 
the measurement problem. But it should be possible to use current quantum tech¬ 
nology to engineer microscopic systems made of a modest number, m of quits or 
electrons, which by their combinatorial complexity cannot be expected to have any 
natural copies. These would be microscopic classical systems. It should be possible 
to recognize them by their spectra. 

• N = 2. If we can engineer a single such microscopic classical systems, we can make 
two or several of them. A single pair would have no quantum potential, as that is a 
three body interaction. 

• N = 3. At this point the quantum potential enters. This should have consequences 
which are easily testable. 

With i = 1, 2, 3 we have, in one spatial dimension. 


U‘«) 


*L{( 1 

8m x \ZT(3, 2) 


D( 2,1)) + (^( 1,3) 


D( 3,2)) + (d(2,1) 


1 

^( 1 , 3 ) 


} 


(56) 


• Modest N. If we could study a sequence of experiments from N = 1 to relatively 
large N we could see the transition between classical behaviour for N = 1 and 
quantum dynamics for large N. 

• Non-linear corrections to the Schroedinger equation. We see that the Schroedinger 
equation receives corrections from terms such as U AV in (). 

This leads to a modified Hamilton-Jacobi equation 


where 


S = 1 q ab { — )( — 
2 m V dx aa V dxba 


+ u-—^-^ + au q 

2m y/p 


A U Q = 


r' 2 d k 2 


Ndd-\-2 2m 


VV _ 2 W _ 2 (V»(V <t V 2 p) 


P 


P 


P 


(57) 

(58) 
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for some length scale r'. This implies non-linear modifications of the Schroedinger 
equation 

V 2 + U + A U Q (^)] T (59) 

at \ 2 m ) 

Notice that probability conservation is unaffected. 

To first order this perturbs energy eigenvalues 

A E = [ d d z^AU Q (^)^> (60) 


Such terms are very well bounded by experiment j3T| . 
It is easy to estimate that 


so 


A E 


1 r' 2 ft 2 
iV§ «o 2 mal 


R 2 h 2 

al 2ma^ 


A E _ 1 r' 2 
E ~ ivi Oq 


(61) 

(62) 


We learn an important lesson from this, which is that the infrared cutoff scale R sets 
the size of the expected departures from linear quantum dynamics. In our present 
treatment this is a free parameter, thus we can look for possibilities to bound it by 
experiment. 


5 Comments and objections 

5.1 Quantum statistics 

It is straightforward to derive the statistics of bosons and fermions from the assumptions 
enunciated above. We require only a translation into the present notation of the standard 
derivations. 

Suppose the configuration space refers to M identical particles in d dimensions, whose 
positions are given by x a Ik and which have phase variables «;/, (one phase for each con¬ 
figuration of the M particles.). Here / = 1,... M refers to the positions of the identical 
particles, where as k — 1,... N refers to the members of any additional ensemble these 
are members of. 

As application of the PSR we can ask why the world is as it is rather than with the 
positions of the first and second particles switched. 


rp® 1 A _V ry®' 

•^k 1 •'k 


&2> 


(63) 
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By saying that the particles are identical we mean that there are no differences, i.e. the 
system is defined for each member of the ensemble by the unordered set of positions 


f ry> a rp 0, 

\ x kli x k2-> 

<3 Jfi 

<N 
<3 -Sfi 

III 


(64) 

It follows that the probability densities satisfy 



p{ x l 

■,t)= p{x a k 2 ,x a kl , 


(65) 

for all time, which implies also 




Pi X klJ X k2i • • 

■ 1 P(. X k 2) X kh 


(66) 

It follows directly from (150b and (51) that the phases 

satisfy 


S( x kli ■ ■ ■ > 

t ) = S(x a k 2 ,x a kl ,. 

■ , t) + 0 

(67) 


where 0 is a constant phase. But recalling (fT6b we see by doing the switch twice that 20 
must be zero or a multiple of 2 tt. Hence we have under the switch of two particles 


w k -t ±w k (68) 

which gives us bosonic and fermionic statistics. 

It is important to make an additional point. Consider a single system with a large 
number, M, of identical particles moving in the same external potential. These might be 
helium atoms in a beaker of superfluid helium or electrons in a doped semiconductor. 
These constitute an ensemble of identical particles in isomorphic potentials, which hap¬ 
pens, in this case, to be the same potential. Hence the inter-ensemble interactions may 
be expected to be active here as well. Hence, even if there are no copies of that beaker or 
that precise doped semiconductor in the universe, we already are dealing with large M's 
greater than 10 20 . Hence quantum mechanics may be expected to hold well in these cases. 
Indeed, nothing in the derivation of quantum statistics we have just given precludes the 
imposition of the inter-ensemble interaction between the identical particles. 

Hence, the theory we have described here will agree with quantum mechanics for 
cases like these of so-called "macroscopic" quantum systems, because they constitute an 
ensemble of microscopic systems all by themselves. 

5.2 Preferred simultaneities 

It may be objected that the present formulation, by virtue of its invoking interactions 
between distant subsystems, requires a preferred simultaneity. Does this inhibit its appli¬ 
cation to relativistic systems? 

One may note that the same criticism may be made to any completion of quantum 
mechanics which gives a more completed description of the trajectories of individual sys¬ 
tems. We know this because of a theorem of Valentini li29]l . 
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One answer is that general relativity has recently been reformulated as a theory with 
preferred foliations. Called shape dynamics 113011 , this formulation trades many fingered 
time, or refoliation invariance, for local scale invariance. Shape dynamics reproduces all 
the predictions of general relativity which have been confirmed, hence our knowledge of 
space-time physics is consistent with the existence of a preferred foliation. 

5.3 Defining degrees of relational similiarity 

We use informally two notions, similarity of causal pasts of events and similarity of recent 
pasts, or preparations, of isolated subsystems and implied they are related to each other. 
The first was defined in a causal set ontology, the second within an operational framework 
describing subsystems. More work needs to be done defining each of these notions and 
their relationship to each other. 

In this further work, the absolute notions of similar or not can be replaced by degrees 
of similarity. This would be incoherent in a fundamental theory, but it is important to 
emphasize that the task we are engaged with is the construction of a theory of subsystems, 
which is necessarily approximate. (This is related to the cosmological dilemma discussed 
in 111130.) 

6 Motivations 

The hypothesis behind this completion of quantum theory are inspired by a broad princi¬ 
ple, which has already had enormous influence on our understanding of space and time. 
This is Leibniz's Principle of sufficient reason (PSR)|4j, which can be stated as 

• Every question of the form of why is the universe like X rather than Y has a reason 
sufficient to explain why. 

Big philosophical principles function in science as guides; in this spirit we may take 
the PSR as the aspiration to eliminate arbitrary choices from the statements of the laws and initial 
conditions of physics. This aspirational version of the PSR|j3HT]] has been very influential in 
the search for fundamental laws. Some of the ideas it has inspired are: 

• Space and time are relational. Space and time represent relational and dynamic 
properties that allow each subsystem of the universe to be uniquely distinguished 
in term of their relations to the rest of the universej5|. This was the basis for the cri¬ 
tique, made by LeibnizfU, Mach|6j and others, of Newton's conception of absolute 
space, which inspired Einstein in the construction of its first full realization in his 
general relativity theory |7| . 

• This has further consequences which we exploited in this paper. Localization in space 
is a consequence of having unique relational properties, i.e. a unique causal neighbourhood. 
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Objects or subsystems that are hard to distinguish from similar systems should be hard to 
localize unambiguously, and thus may be in causal contact. 

• The laws of physics have no ideal elements and depend on no fixed, non-dynamical 
structures(8|. This is the basis of the requirement that fundamental theories be back¬ 
ground independent, which is satisfied by general relativity in the cosmological case 
in which space-time is spatially compact. 

• Einstein's principles of causal closure and reciprocity Everything that influences the 
evolution of a subsystem of the universe is itself a part of the universe. There are no 
entities which effect the evolution of degrees of freedom, which do not themselves 
evolve in response to influences. If an ensemble of systems influences a system then every 
last member of that ensemble must exist as a physical system somewhere in the universe. 

• One of the most important implications of the PSR is another principle, known as 
the principle of the identity of the indiscernible. (PII) This states that any two events or 
subsystems of the universe which are distinct have distinct properties derived from 
their relations with the rest of the universe |[4|. 

The PII implies that fundamental, cosmological theories have no global symmetries, 
because a global symmetry of a cosmological theory would be a transformation be¬ 
tween distinct states of the universe, each of which has exactly the same relational 
properties. Indeed, general relativity in the cosmological case has no global symme¬ 
tries or non-vanishing conservation laws|[9l. 

Global symmetries arise within effective theories of subsystems of the universe, and 
they describe transformations of a subsystem with respect to the rest of the uni¬ 
verse which is, for purposes of the effective description, regarded as a fixed, non- 
dynamical, frame of reference f3j[l). 

• Maximal variety We have seen how the principle of maximal variety is realized 
dynamically by incorporating the negative of the variety as a potential energy. This 
realizes the PII dynamically, by acting to make systems which are similar distinct. 

To illuminate the idea of variety we can describe it in the context of a causal setflTf. 
We start by defining the n'th neighbourhood of an event, I, called it consists 

of the subsystem consisting of all events n causal links into the future or past from I. 
Then we can call the distinguishability of two events, I and J, T>(I, J ) = where 
rijj is the smallest n such that J\f n (I) is not isomorphic to M n {J). The higher V is, the 
less effort it is to distinguish every event from every other by describing their causal 
neighbours. 

In flOl . Julian Barbour and I proposed two uses of the concept of the variety of a 
system. First, variety measures the complexity or the organization of a system. We 
showed several examples in which V provides an interesting measure of complexity. 
For example, the variety of a city is a measure of how easy it is to know where you 
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are by looking around. A modern suburban development has lower variety than an 
old city because it has more corners from which the view is similar. 

But we also proposed a dynamical principle that the universe evolve so as to ex- 
tremize its variety. We speculated that this highly non-local dynamical principle 
might underlie quantum theory. In this paper we develop this idea by showing that 
the quantum potential of Bohm can be understood to be a measure of the variety of 
a system of similar subsystems of the universe. 

• The PSR also demands that we can explain how and why the particular laws which 
describe local physics in our universe were selected from a large set of equally con¬ 
sistent laws lfl2l Hll2ll3H. As the American philosopher Charles Sanders Peirce enun¬ 
ciated in the 1890's this can only be done in a way that yields falsifiable predictions 
if the laws are not absolute but are the result of a dynamical process of evolution |[T3l 

• We also argue in [TJ, [2} (3j that neither quantum nor classical mechanics mechanics 
can be usefully extended to a theory of the whole universe. One of several reasons 
is that any such extension leaves unanswered the questions of why the laws of na¬ 
ture and the cosmological initial conditions were chosen, thus the PSR is unfulfilled. 
This means that quantum mechanics is restricted to a theory of subsystems of the 
universe, and it must theresor be an approximation to a cosmological theory which 
does not allow a free specification of laws and initial conditions. In this paper we 
seek to build on this insight by constructing quantum mechanics explicitly as a the¬ 
ory of subsystems. 

6.1 Taking the principle of the identity of the indiscernible seriously 

Aa we seek to apply these ideas to quantum physics we should be mindful that all serious 
approaches to quantum gravity agree that space is emergent. The emergence of space is 
a contingent property of a phase that the universe may be in. But if space is emergent, 
then locality is emergent too. This implies that how physics sorts itself out into a mix of 
local and non-local, in which strictly local propagation of information and energy takes 
place in a sea of non-local quantum correlations, must be a result of a dynamical equilib¬ 
rium characterizing the low level excitations of the phase of the universe in which space 
emerges. 

But if locality is emergent we expect defects or dis-orderings of locality Il5ll . These 
would be represented by pairs or sets of particles which are far from each other in the 
classical emergent metric geometry, but which are actually nearby or adjacent in the real, 
fundamental causal structure. It has been suggested before that this kind of disordered 
locality could be connected to quantum non-locality HT6| , here we propose a novel ex¬ 
pression of this idea that connects it with the previous idea, which suggests that defects 
in localization should be consequence of failures to uniquely distinguish subsystems or 
events, in terms of their role in the dynamical network of relational properties. 
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Furthermore, recent work suggests that the emergent locality is relative IfTTll . In past 
work we understood this to mean that different observers, at different places and in dif¬ 
ferent states of motion ascribe different notions of locality to distant events. Here we 
further relativize and radicalize the notion of locality by understanding that locality is a 
consequence of identity, so that only subsystems which may be uniquely identified by the 
network of interactions and relationships they participate in get localized uniquely in a 
conventional way 

Thus, if locality is a consequence of distinctiveness, as measured by the relations of 
an event or subsystem with the rest of the universe, we expect two subsystems which are 
very similar to each other to be near to each other, in the true microscopic causal structure. 
Subsystems which are similar to each other in the sense of having nearly isomorphic 
relations to neighbouring events, may then be able to interact with each other, in spite 
of being far away from each other in the emergent spatial geometry. In this paper we 
propose a form for such non-local interactions, which acts to increase the distinctiveness 
of pairs of subsystems. This is necessary to prevent violations of the PII. We have show in 
this paper that this gives rise exactly to Bohm's quantum potential, and hence to quantum 
mechanics. The Schroedinger equation then emerges as a consequence of a dynamical 
implementation of the PII. That principle is then re-interpreted, not as an epistemological 
truism, but as a dynamical principle that underlies and explains "why the quantum." 

We then saw that the further application of the PII to a system of identical particles 
then gives rise to either fermionic or bossing statistics. 

It then appears that the main features of quantum physics, the statistical, indetermi¬ 
nate character of local laws, and their interruption by non-local correlations, are conse¬ 
quences of the principle of the identity of the indiscernible. We give further arguments 
which support this conclusion. 

6.2 The statistical character of local physics is a consequence of the PII 

We may apply the principle of the identity of the indiscernible to show that local physics 
must be indeterminate on the level of fundamental systems and events. This is an argu¬ 
ment which appeared first in Hl8ll . 

To make this argument, we work with a causal set ontology according to which what 
is real is a thick present of events and processes, which create novel events from present 
events. This is discussec0in more detail in [fT8lH9l . For the construction of non-relativistic 
quantum mechanic below, we will work, in the next sections, in a more operational frame¬ 
work. 

By the PII, two distinct events e and / must have different (that is non-isomorphic) 
causal neighbourhoods. The causal neighbourhood of an event, e, labeled iV(e), is the 
subset of the causal set of events making up the history of the universe which involve e. 

3 For a different approach to causal sets, see (20ll . 
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N(e) is the disjoint union of a past set P(e) and a future set F(e). Now it follows from 
the PII that if there are two distinct events e and / such that their past sets are isomorphic 
then their future sets cannot be isomorphic. 

P(e) = P(f) F(e) ? F(f) (69) 

The same holds true for e and / any subsets of a causal set, such as a subset of an anti 
chain (i.e. a space like region.) 

To give this force in a large universe we can add a bit more structure. Let P(e) n and 
F(e) n be the causal past and future n steps into the past or future. Then we an replace () 
by the requirement that there exists an n much less than Njj, the number of past events in 
the universe, such that 

P(e)n = P(f)n F(e) n ± F(f) n (70) 

In a vast universe, for fixed n < N v there are bound to be instances of P(e) n = P(f) n 
holding sufficiently far into their pasts, because if interactions are simple, there will not 
be that many possibilities for the recent past of an elementary event. 

But, normally F(e) ^ F(f ) will be enforced because the causal past of F(e), denoted 
P[F(e)\, is distinct from P[F(f)] because if one goes far enough in the future the pasts of 
the futures of the two events become distinct. This is because, for most events, the sizes 
of the sets P(e) n and F{e) n grow like n d , where d is the spatial dimension. 

However there are systems that this doesn't apply to, which are isolated systems. These 
are systems that have a restricted causal future that does not grow faster than linearly with 
n, as the system evolves to the future (or, similarly, into the past.). Systems maybe isolated 
naturally, by being sufficiently separated or shielded from their environments. We also 
construct isolated systems in order to focus experiments on fundamental interactions; to 
isolate a system is the basic method of laboratory science. 

It follows that two isolated systems e and / with similar environments and similar 
causal pasts, P(e) ~ P(f) are in danger of violating the PII. Indeed if the laws of physics 
are deterministic, it is exactly those isolated systems which have P(e) = P(f) which we 
would expect, by determinism to evolve identically such that F(e) = F(f). 

So exactly how can two isolated systems avoid violating the PII? First, if the laws of 
nature are statistical and indeterminate, so P(e) = P(f) need not imply F(e) = F(f), 
even if the systems are truly isolated. This argument was developed in []. This, I would 
propose is the origin of quantum statistics. 

But this turns out to be not sufficient, because even if the laws are statistical, so that 
e with past P{e) can have several possible futures, F(e)j, in a big universe with a vast 
number of events there still will arise by chance two distinct events e and / with identical 
causal pasts and identical causal futures. 

Thus, to ensure the PII something more is needed. This is an interaction between 
two isolated systems, with identical pasts that will prevent their having identical futures. 
This interaction has to be repulsive (when expressed in terms of relational observables), to 
ensure that distinct microscopic subsystems have distinct values of their beables, in order 
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to prevent violations of the PII. Here we have proposed that the variety of the ensemble 
be used to generate this inter-ensemble interaction. We saw that this is the origin of the 
quantum potential, of deBroglie-Bohm theory. 

Thus, we arrive at an ensemble interpretation of the quantum state, but the ensemble 
is real and not imagined; it is a real physical ensemble consisting of a finite set of similar 
systems which exist throughout the universe. The ensemble does, as in dBB and Nelson, 
influence the individual member, but that is in accord with the principle of causal closure 
because that influence is just a summary of a great many multi body interactions amongst 
members of an isolated subsystem's ensemble. 

This scheme is non-local, as we know any realist completion of quantum mechanics 
must be. It is indeed wildly non-local, in order to enforce the PII in a huge universe with 
vast numbers of nearly identical elementary systems. But we should not be surprised 
because we know from diverse studies of and approaches to quantum gravity that space 
and locality are expected to be emergent from a more fundamental level of description in 
which they play no role. 

6.3 How the measurement problem is solved 

We see that, in a world governed by the PII, locality is a consequence of having a unique 
identity, and that is a property enjoyed only by systems large and complex enough that 
they have neither copies in the universe nor near copies. There is in this scheme a natural 
definition of macroscopic: if a system is large and complex enough to have no copies (in 
a precise sense defined below), it is not part of any ensemble of subsystems. It is unique 
on its own and hence it neither interacts with, nor an it be confused with, distant similar 
subsystems. Consequently it can be stably localized. 

Such a macroscopic subsystem does not, by its uniqueness, suffer any quantum ef¬ 
fects. It's motion is subject only to local forces, it does not answer to any inter-ensemble 
interactions, hence its center of mass coordinates evolve according to the laws of classical 
mechanics, without a quantum potential. That is to say, cats do not have ensembles of 
similar systems, and they are either dead or alive. This is then the answer to the measure¬ 
ment problem ll2TI . 


7 Conclusions 

The real ensemble hypothesis UZTIl has been strengthened by the use of a greatly simplified 
interaction between members of the ensemble of similar systems, based on the principle 
of extremal variety llTOl . 
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